Automated Detection of Breathing Disturbances

ABSTRACT

Approaches to determining a sleep fitness score for a user are provided, such as may be based upon monitored breathing disturbances of a user. The system receives user state data generated over a time period by a combination of sensors provided via a wearable tracker associated with the user. A system can use this information to calculate a sleep fitness score, breathing disturbance score, or other such value. The system can classify every minute within the time period as either normal or atypical, for example, and may provide such information for presentation to the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 62/547,710, filed Aug. 18, 2017, and entitled “AutomaticDetection of Breathing Disturbances,” which is hereby incorporatedherein by reference in its entirety for all purposes.

TECHNICAL FIELD

This disclosure relates to the field of wearable devices, andparticularly to techniques for detecting breathing-related issues usingone or more sensors of such devices.

BACKGROUND OF THE DISCLOSURE

Wearable electronic devices have gained popularity among consumers. Awearable electronic device may track a user's activities using a varietyof sensors. Data captured from these sensors can be analyzed in order toprovide a user with information that can help the user to maintain ahealthy life style. In order to determine information about a user'sactivities, a conventional wearable electronic device collects activitydata and runs computations on that data. Difficulty is encountered inobtaining accurate determinations of a user's activities, however, asthese wearable electronic devices, being made to be worn by a user, aretypically packaged in a compact casing with limited internal space. Thislimited space results in the devices containing less powerfulcomponents, such as processors, than would be possible in largerelectronic devices. This limits the ability of these devices to runcomplex computations or perform complex operations. Further,conventional wearable electronic devices are subject to considerableincidental movements and noise, which can impact the inclusion or use ofvarious components in these devices.

Wearable electronic devices can track metrics related to particularactivities, such as a step count metric for running and walkingactivities. Other metrics that may be tracked by a wearable electronicdevice include metrics related to sleep. Typically, to initiate trackingof sleep metrics, a wearable electronic device provides an interfaceenabling the user to provide a manual indication that the user plans togo to sleep. For example, the wearable electronic device may include aphysical or digital button that the user can push or tap to indicatethat he or she is about to go to sleep. The timing of this action may besignificantly different than when the user actually sleeps, and does notaccount for periods when the user is awake unless the user manuallyindicates that the user is now awake.

BRIEF SUMMARY OF THE DISCLOSURE

Approaches in accordance with various embodiments provide for themonitoring of breathing data for a user. Such information may beobtained by a monitoring system or device, and can be used to determinevarious patterns, trends, or aspects of the user's breathing, which canbe indicative of breathing disturbances or other irregularities of auser, such as during a period of sleep.

Such a system may include a wearable activity tracker, or other wearableelectronic device, configured to be worn on a wrist or other part of auser's body. The device can be in communication with a separatecomputing device, such as a smart phone or other mobile computingdevice, configured to execute a mobile application to process or collectat least a portion of the user data. The computing device and/or trackercan also be in communication with at least one server or remotecomputing system in various embodiments, such as to offload at leastsome of the processing or persistently store the user or result data.

A wearable activity tracker in at least one embodiment may include: atleast one green photoplethysmographic (PPG) sensor configured togenerate first PPG data; at least one red PPG sensor configured togenerate second PPG data; at least one infrared PPG sensor configured togenerate third PPG data; at least one accelerometer configured togenerate acceleration data useful for determining motion; and at leastone processor configured to generate a set of features based on thefirst PPG data, the second PPG data, the third PPG data, and theaccelerometer data. The mobile application, when executed by the mobiledevice, may configure the mobile device to receive the set of featuresfrom the wearable activity tracker and wirelessly transmit the set offeatures to a server. The server may be in communication with the mobiledevice and further configured to: receive the set of features from themobile device; determine, based on a first subset of the set of featuresrelating to the accelerometer data, a sleep period during which the userof the wearable activity tracker was asleep; for each respective minuteof a plurality of minutes within the sleep period, classify therespective minute as one of (i) a normal minute or (ii) a breathingdisturbed minute, based on a second subset of the set of featurescorresponding to the first, second, and third PPG data generated duringthe respective minute and data corresponding to a time period outsidethe sleep period; generate a breathing disturbance score (or otherbreathing-related score such as a sleep fitness score) for the sleepperiod based on the classification of the plurality of minutes withinthe sleep period; and update a benchmark breathing disturbance scoreassociated with a user category corresponding to the user based on thegenerated breathing disturbance score. The mobile application mayfurther configure the mobile device to receive the determined score(s)from the server and cause the score(s) to be displayed on a displayscreen of the mobile device, or otherwise made available to a respectiveuser, along with the updated benchmark breathing, sleep fitness, orother such score associated with the user category.

Such a system can have any sub-combination of the above-mentioned andother device sensors. In some embodiments, at least a subset of thegreen, red, and/or infrared PPG sensors, as well as theaccelerometer(s), are located on the same side of the user's wrist whenthe wearable activity tracker is worn on the wrist of the user. Thesystem can include a sensor, such as a bed sensor, that is configured togenerate respiration data indicative of the user's respiration rate(additionally or alternatively, heart rate and heart beat-to-beatinterval), wherein the server can further be configured to classify eachrespective minute based on the respiration data. The server can beconfigured to maintain past breathing disturbance scores for a pluralityof users each associated with respective wearable activity trackers,which can be used for comparison purposes in at least some embodimentsand where such usage of the data is permitted. At least one processor ofthe wearable activity tracker can also be configured to determine whenthe accelerometer data indicates at least a threshold level of movementfor a threshold amount of time, and to deactivate at least some of theat least one green PPG sensor, the at least one red PPG sensor, the atleast one infrared PPG sensor.

Approaches in accordance with various embodiments can facilitatemonitoring of breathing disturbances of a user of a wearable device. Oneexample approach can include receiving sensor data generated by one ormore sensors of the wearable device worn by the user, the one or moresensors comprising at least one accelerometer, the sensor datacomprising accelerometer data indicative of movement of the user. Basedat least in part on the accelerometer data, a sleep period can bedetermined that corresponds to a portion of the accelerometer dataindicative of movement that is consistent with the user sleeping. Foreach respective temporal window of a plurality of temporal windowswithin the sleep period, the respective temporal window can beclassified, such as for one of (i) breathing disturbed or (ii) breathingnot disturbed (or regular), based on the sensor data generated duringthe respective temporal window and data corresponding to a time periodoutside the sleep period. A breathing disturbance score, sleep fitnessscore, or other such value can be determined based on the classificationof the plurality of temporal windows, with the score(s) being output forpresentation to the user.

Approaches in accordance with various embodiments can have anysub-combination of the following features: where the method furtherincludes causing the breathing disturbance score of the user to berendered on a display screen of a mobile device wirelessly paired withthe wearable device along with a benchmark disturbance scorecorresponding to the user; where the method further includes determininga percentage of temporal windows classified as breathing disturbed outof the plurality of temporal windows, and selecting the breathingdisturbance score from a plurality of breathing disturbance scores basedon the percentage; where the method further includes receivingadditional sensor data generated by at least one of the one or moresensors of the wearable device, wherein the additional sensor data isgenerated during a second period outside the sleep period, determining abaseline physiological metric associated with the user based on theadditional sensor data, and classifying said each respective temporalwindow further based on the baseline physiological metric associatedwith the user; where the method further includes determining, based onthe sensor data, that the sleep period is associated with movement lessthan a threshold level and that a second period outside the sleep periodis associated with movement greater than the threshold level, andclassifying one or more temporal windows within the sleep withoutclassifying any temporal window within the second period outside thesleep period; where the method further includes causing an indication ofhow each temporal window within the sleep period is classified to bedisplayed in a user interface such that the user interface does notinclude an indication of how one or more temporal windows within thesecond period are classified; where the method further includesoutputting the breathing disturbance score based on a determination thatthe sleep period is longer than a threshold amount of time; where themethod further includes determining respiration rate data for the sleepperiod based on the sensor data, and classifying a first temporal windowof the plurality of temporal windows as breathing disturbed based on adetermination that a portion of the respiration rate data correspondingto the first temporal window includes a pattern of low values followedby a spike in value; where the sensor data includes first sensor datagenerated by at least one red PPG sensor and second sensor datagenerated by at least one green PPG sensor, the method further includingincreasing a first weight assigned to the first sensor data based on adetermination that the second sensor data has not changed more than athreshold amount for a threshold amount of time; where the sensor dataincludes first sensor data generated by one or more PPG sensors andsecond sensor data generated by one or more motion sensors, the methodfurther including classifying said each respective temporal window basedon both of the first sensor data and the second sensor data; and wherethe sensor data includes first sensor data generated by a red PPG sensorand second sensor data generated by an infrared PPG sensor, the methodfurther including extracting a low-frequency component of the firstsensor data and a low-frequency component of the second sensor data,determining processed sensor data based on dividing the low-frequencycomponent of the first sensor data by the low-frequency component of thesecond sensor data, and classifying at least one of the plurality oftemporal windows based on the processed sensor data.

One aspect of the disclosure provides non-transitory physical computerstorage storing instructions. The instructions may, when executed by oneor more processors, configure the one or more processors to: receivesensor data generated by one or more sensors of a wearable device wornby a user, the one or more sensors comprising at least oneaccelerometer, the sensor data comprising accelerometer data indicativeof movement of the user; determine, based on the accelerometer data, asleep period corresponding to a portion of the accelerometer dataindicative of movement that is consistent with the user sleeping; foreach respective temporal window of a plurality of temporal windowswithin the sleep period, classify the respective temporal window as oneof (i) breathing disturbed or (ii) not breathing disturbed, based on thesensor data generated during the respective temporal window and datacorresponding to a time period outside the sleep period; determine abreathing disturbance score based on the classification of the pluralityof temporal windows; and output the breathing disturbance score forpresentation to the user.

Such a non-transitory physical computer storage in accordance withvarious embodiments can have any sub-combination of the followingfeatures: where the instructions, when executed by the at least oneprocessor, further configure the at least one processor to classify saideach respective temporal window further based on a determination that aportion of the accelerometer data corresponding to the respectivetemporal window is indicative of movement less than a threshold level;where the instructions, when executed by the at least one processor,further configure the at least one processor to wirelessly transmit thebreathing disturbance score to a cloud service for storage inassociation with the user; and where the instructions, when executed bythe at least one processor, further configure the at least one processorto receive a plurality of past breathing disturbance scores associatedwith the user from the cloud service, and to render a history ofbreathing disturbance scores on a display screen of a mobile devicewirelessly paired with the wearable device based on the plurality ofpast breathing disturbance scores received from the cloud service.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments described herein are illustrated by way of example, andnot by way of limitation, in the figures of the accompanying drawings inwhich like references indicate similar elements.

FIG. 1A is a block diagram illustrating certain components of an examplewearable device in accordance with aspects of this disclosure.

FIG. 1B is a block diagram illustrating example sensors which may be incommunication with a processor of a wearable device in accordance withaspects of this disclosure.

FIG. 1C is an example block diagram of a system used for determining aheart rate in accordance with aspects of this disclosure.

FIG. 2 illustrates an overview of a system for monitoring breathingdisturbances in accordance with aspects of this disclosure.

FIG. 3 illustrates an overview of the signal processing performed by thesystem of FIG. 2 in accordance with aspects of this disclosure.

FIG. 4 illustrates a receiver operating characteristic (ROC) curve of asystem in accordance with aspects of this disclosure.

FIG. 5 illustrates an example method of monitoring breathingdisturbances in accordance with aspects of this disclosure.

FIG. 6 illustrates another example method of monitoring breathingdisturbances in accordance with aspects of this disclosure.

FIG. 7 illustrates an example method of determining a fitness score inaccordance with aspects of this disclosure.

DETAILED DESCRIPTION

Approaches in accordance with various embodiments can obtain and analyzehealth-related information for a user wearing, or otherwise utilizing, adevice such as a wearable tracking device. A tracking device in someembodiments can include one or more sensors for tracking of an activityperformed by a user of a wearable device. Some example wearable devicesmay be referred to as wearable activity trackers or wearable activitytracking devices. Various algorithms or techniques for trackingactivities have been developed, and these algorithms may be specializedbased on the type of activity performed by the user. For example, theuser may wear the wearable device all day, and for each activityperformed by the user throughout the day and night, a specializedalgorithm may generate and present relevant activity metrics to theuser. For example, the wearable device may collect sensor data from oneor more sensors provided on the wearable device while the user issleeping, and generate and present information such as a sleep scoreindicative of how well the user is sleeping each night.

FIG. 1A is a block diagram illustrating an example wearable device inaccordance with aspects of this disclosure. The wearable device 100 mayinclude a processor 120 (which may be a set of processors), a memory130, a wireless transceiver 140, and one or more sensor(s) 160. Thewearable device 100 may also optionally include a user interface 110 andone or more environmental sensor(s) 150. The wireless transceiver 140may be configured to wirelessly communicate with a client device 170and/or server 175, for example, either directly or when in range of awireless access point (not illustrated) (e.g., via a personal areanetwork (PAN) such as Bluetooth pairing, via a wireless local areanetwork (WLAN), via wide area network (WAN), etc.). Each of the memory130, the wireless transceiver 140, the one or more sensor(s) 160, theuser interface 110, and/or the one or more environmental sensor(s) 150may be in electrical communication with the processor 120. The clientdevice 170 may be a smartphone, a tablet, or another mobile deviceexecuting software (e.g., a mobile application) configured to performone or more techniques described herein. The server 175 may beimplemented using one or more computing devices executing softwareconfigured to perform one or more techniques described herein. Thetechniques described herein may be performed by the wearable device 100,the client device 170, and the server 175 in a distributed manner. Thewearable device 100 (also referred to herein as wearable activitytracker) may be any of a smartwatch, a watch, a wrist-wearable fitness-,health-, or activity-monitoring device, and the like, although theconcepts described herein may be implemented on any type of portable orwearable devices including one or more sensors.

The memory 130 may store instructions for causing the processor 120 toperform certain actions. In some embodiments, the sensors 160 mayinclude one or more of biometric sensors, optical sensors (e.g., aphotoplethysmographic (PPG) sensor), motion sensors or inertial sensors(e.g., accelerometer, gyroscope, digital compass, etc.), barometricsensors (e.g., altimeter, etc.), geolocation sensors (e.g., GPSreceiver), and/or other sensor(s). Further information regarding suchsensors are described in more detail below (e.g., in connection withFIG. 1B).

The wearable device 100 may collect one or more types of physiologicaland/or environmental data from the one or more sensor(s) 160, the one ormore environmental sensor(s) 150, and/or external devices andcommunicate or relay such information to other devices (e.g., the clientdevice 170 and/or the server 175), thus permitting the collected data tobe viewed, for example, using a web browser or network-basedapplication, such as for an individual account or shared account, orshared via social media where permitted or approved by the user. As usedherein, the term “collect,” in addition to having its ordinary meaning,may be used interchangeably with “determine,” “extract,” “calculate,”“generate”, etc. to refer to the steps performed to arrive at thedesired data (e.g., breathing disturbance metrics). For example, whilebeing worn by the user, the wearable device 100 may perform biometricmonitoring via calculating and storing the user's step count using theone or more sensor(s) 160. The wearable device 100 may transmit datarepresentative of the user's step count to an account on a web service(e.g., fitbit.com), computer, mobile phone, and/or health station wherethe data may be stored, processed, and/or visualized by the user. Thewearable device 100 may measure or calculate other physiologicalmetric(s) in addition to, or in place of, the user's step count. Suchphysiological metric(s) may include, but are not limited to: energyexpenditure, e.g., calorie burn; floors climbed and/or descended; heartrate; heartbeat waveform; heart rate variability; heart rate recovery;respiration, oxygen saturation (SpO₂), blood volume, blood glucose, skinmoisture and skin pigmentation level, location and/or heading (e.g., viaa GPS, global navigation satellite system (GLONASS), or a similarsystem); elevation; ambulatory speed and/or distance traveled; bloodpressure; blood glucose; skin conduction; skin and/or body temperature;muscle state measured via electromyography; brain activity as measuredby electroencephalography; weight; body fat; caloric intake; nutritionalintake from food; medication intake; sleep periods (e.g., clock time,sleep phases, sleep quality and/or duration); pH levels; hydrationlevels; respiration rate; and/or other physiological metrics.

The wearable device 100 may also measure or calculate metrics related tothe environment around the user (e.g., with the one or moreenvironmental sensor(s) 150), such as, for example, barometric pressure,weather conditions (e.g., temperature, humidity, pollen count, airquality, rain/snow conditions, wind speed), light exposure (e.g.,ambient light, ultra-violet (UV) light exposure, time and/or durationspent in darkness), noise exposure, radiation exposure, and/or magneticfield. Furthermore, the wearable device 100 (and/or the client device170 and/or the server 175) may collect data from the sensor(s) 160and/or the environmental sensor(s) 150, and may calculate metricsderived from such data. For example, the wearable device 100 (and/or theclient device 170 and/or the server 175) may calculate the user's stressor relaxation levels based on a combination of heart rate variability,skin conduction, noise pollution, and/or sleep quality. In anotherexample, the wearable device 100 (and/or the client device 170 and/orthe server 175) may determine the efficacy of a medical intervention,for example, medication, based on a combination of data relating tomedication intake, sleep, and/or activity. In yet another example, thewearable device 100 (and/or the client device 170 and/or the server 22)may determine the efficacy of an allergy medication based on acombination of data relating to pollen levels, medication intake, sleepand/or activity. These examples are provided for illustration only andare not intended to be limiting or exhaustive.

FIG. 1B is a block diagram illustrating a number of example sensors thatmay be included in the wearable device 100 in accordance with aspects ofthis disclosure. For example, in the embodiment of FIG. 1B, the wearabledevice 100 includes an accelerometer 161 (e.g., a multi-axisaccelerometer), a gyroscope 162, a magnetometer 163, an altimeter 164, aGPS receiver 165, a green PPG sensor 166, a red PPG sensor 167, aninfrared (IR) PPG sensor 168, and one or more other sensors 167(including but not limited to, e.g., a temperature sensor, an ambientlight sensor, a galvanic skin response (GSR) sensor, a capacitivesensor, a humidity sensor, a sound sensor, a force sensor, a multi-axisaccelerometer, a gravity sensor, a piezoelectric film sensor, a rotationvector sensor, etc.), all of which may be in communication with theprocessor 120. Each of the sensors illustrated in FIG. 1B may be inelectrical communication with the processor 120. The processor 120 mayuse input received from any combination of the sensors in detecting thestart of an exercise and/or in tracking the metrics for the exercise.One or more of the sensors described herein may not be within thewearable device 100. For example, these sensors may be placed on thechest of the user, the mattress or bedside table of the user, while thewearable device 100 is worn by the user.

Although the example of FIG. 1B illustrates sensors 161-169, in otherembodiments the wearable device 100 may include a fewer number ofsensors and/or any other subsets and combinations of the sensors (e.g.,only the accelerometer 161, a combination of the accelerometer 161, redPPG sensor 167, and IR PPG 168, a combination of the accelerometer 161and the green PPG sensor 166, etc.). The wearable device 100 may alsoinclude one or more additional sensors not illustrated in FIG. 1B.

Additionally, in some implementations the GPS receiver 165 may belocated in the client device 170 rather than the wearable device 100. Inthese implementations, the processor 120 may wirelessly communicate withthe client device 170 to control and/or receive geolocation data fromthe GPS receiver 165 and/or other geolocation sensor(s).

In related aspects, the processor 120 and other component(s) of thewearable device 100 (e.g., shown in FIGS. 1A and 1B) may be implementedusing any of a variety of suitable circuitry, such as one or moremicroprocessors, application specific integrated circuits (ASICs), fieldprogrammable gate arrays (FPGAs), discrete logic, software, hardware,firmware or any combinations thereof. When the techniques areimplemented partially in software, a device may store instructions forthe software in a suitable, non-transitory computer-readable medium andexecute the instructions in hardware using one or more processors toperform the techniques of this disclosure. In further related aspects,the processor 120 and other component(s) of the wearable device 100 maybe implemented as a System-on-Chip (SoC) that may include one or morecentral processing unit (CPU) cores that use one or more reducedinstruction set computing (RISC) instruction sets, a GPS receiver 165, awireless wide area network (WWAN) radio circuit, a WLAN radio circuit,and/or other software and hardware to support the wearable device 100.

FIG. 1C is an example block diagram of a system used for determiningheart rate in accordance with aspects of this disclosure. As shown inFIG. 1C, the wearable device 100 may include a system 190 of circuitcomponents for determining the heart rate of the user based on anoptical PPG signal (e.g., received from the green PPG sensor 166 of FIG.1B) and a motion signal or signature (e.g., received from theaccelerometer 161 of FIG. 1B). As used herein, a motion signature mayrefer to any biometric signature or signal that may be received fromand/or based on output data from one or more of sensors, such as, forexample, inertial sensor(s) (e.g., accelerometer(s) and gyroscope(s)),barometric sensors(s) (e.g., altimeter(s)), which may be indicative ofthe activity and/or physiological state of a user of the wearable device100. The system 190 may be implemented by hardware components and/or insoftware executed by the processor 120. The system 190 may include firstand second spectra estimators 191 and 192, a multi-spectra tracker 193,an activity identifier or discriminator 194, and a track selector 195.Each of the first and second spectra estimators 191 and 192 may includea Fast Fourier Transform (FFT) block and a peak extraction block. In theexample of FIG. 1C, the activity identifier 194 may use the peaksextracted from the motion signature to determine the activity that theuser is performing (e.g., sedentary, walking, running, sleeping, lyingdown, sitting, biking, typing, elliptical, weight training, swimming,etc.). This determination of the current activity of the user may beused by the multi-spectra tracker 193 and the track selector 195 inextracting the heart rate from the optical PPG signal. Thus, the motionsignature in FIG. 1C may be used by the system 190 to determine thecurrent activity of the user. In other embodiments, the processor 120may use a similar technique to the activity identifier 194 indetermining the type of an exercise, as discussed in greater detailbelow.

The blocks illustrated in FIG. 1C are merely examples of componentsand/or processing modules that may be performed to supplement a PPGsignal with a motion signature to determine heart rate. However, inother implementations, the system 190 may include other blocks or mayinclude input from other biometric sensors of the wearable device 100.

Under certain operating conditions, the heart rate of the user may bemeasured by counting the number of signal peaks within a time window orby utilizing the fundamental frequency or second harmonic of the signal(e.g., via an FFT). In other cases, such as heart rate data acquiredwhile the user is in motion, FFTs may be performed on the signal andspectral peaks extracted, which may then be subsequently processed by amultiple-target tracker which starts, continues, merges, and/or deletestracks of the spectra.

In some embodiments, a similar set of operations may be performed on themotion signature and the output may be used to perform activitydiscrimination which may be used to assist the multi-spectra tracker193. For instance, it may be determined that the user was stationary andhas begun to move. This information may be used by the multi-spectratracker 193 to bias the track continuation toward increasingfrequencies. Similarly, the activity identifier 194 may determine thatthe user has stopped running or is running slower and this informationmay be used to preferentially bias the track continuation towarddecreasing frequencies.

Tracking may be performed by the multi-spectra tracker 193 withsingle-scan or multi-scan, multiple-target tracker topologies such asjoint probabilistic data association trackers, multiple-hypothesistracking, nearest neighbor, etc. Estimation and prediction in thetracker may be done through Kalman filters, spline regression, particlefilters, interacting multiple model filters, etc.

The track selector 195 may use the output tracks from themultiple-spectra tracker 193 and estimate the user's heart rate based onthe output tracks. The track selector 195 may estimate a probability foreach of the tracks that the corresponding track is representative of theuser's heart rate. The estimate may be taken as the track having themaximum probability of being representative of the user's heart rate, asum of the tracks respectively weighted by their probabilities of beingrepresentative of the user's the heart rate, etc. The activityidentifier 194 may determine a current activity being performed by theuser which may be used by the track selector 195 in estimating theuser's heart rate. For instance, when the user is sleeping, sitting,lying down, or sedentary, the user's estimated heart rate may be skewedtoward heart rates in the 40-80 bpm range. When the user is running,jogging, or doing other vigorous exercise, the user's estimated heartrate may be skewed toward elevated heart rates in the 90-180 bpm range.The activity identifier 194 may determine the user's current activity(e.g., a current exercise) based at least in part on the speed of theuser. The user's estimated heart rate may be shifted toward (or whollyobtained by) the fundamental frequency of the selected output track whenthe user is not moving. The output track that corresponds to the user'sheart rate may be selected by the track selector 195 based on criteriathat are indicative of changes in activity. For instance, when the userbegins to walk from being stationary, the track selector 195 may selectthe output track that illustrates a shift toward higher frequency basedon output received from the activity discriminator 194.

Although some embodiments of the present disclosure are described withrespect to heart rate, the techniques described herein may be extendedto other metrics. For example, sensor data generated by the one or moresensors described herein may be used to determine respiration, Sp0₂,blood volume, blood glucose, skin moisture, and skin pigmentation leveland, for example, utilize such metrics for activitydetection/identification.

The motion sensor(s) of the wearable device 100 (e.g., a multi-axisaccelerometer 161, a gyroscope 162, or another motion sensor) maygenerate motion data samples that represent motion for a plurality oftime intervals (e.g., the motion sensor is inside the wearable device100 and the motion data represent motion of the wearable device 100).The motion sensor may generate a number of motion data samples during atime interval. The number of samples generated in a time interval maydepend on the sampling rate of the motion sensor and the length of timeof the interval. In the case that the motion sensor is an accelerometer,the motion data samples may characterize a measurement of accelerationalong an axis of movement. In some cases, a motion data sample may be adata value that consumes a given amount of storage (e.g., 64-bits,32-bits, 16-bits, and so on). The amount of storage consumed by a samplemay depend on the implementation of the motion sensor used in theembodiment and the communication interface used to transfer the samplesfrom the motion sensor to memory accessible by the microprocessor of thewearable device.

The wearable device 100 may comprise one or more optical orelectro-optical sensors, such as, for example, one or more PPG sensors.The PPG sensor(s) of the wearable device 100 (e.g., green PPG sensor166, red PPG sensor 167, IR PPG sensor 168, etc.) may generate PPG datausable to calculate heart rate, heart rate variability, respirationrate, and/or oxygen saturation, among other things. PPG data can useused to calculate a user's heart rate by measuring the time betweenpeaks or by calculating a dominant frequency in the optical signal. Formost users, heart rate drops soon after onset of sleep and continuesdropping over the course of sleep until early in the morning. Heart raterises when the users wake up or during short disturbances during sleep.Measuring heart rate allows the system to better identify periods ofsleep. Further, heart rate is a good indicator to separate periods ofsleep from periods of lying still, which could confuse a motion sensorlike an accelerometer. Some users do not see this characteristic drop inheart rate with sleep. Such users can be identified after they wear thewearable electronic device for a few days and a different algorithm canbe used on such users.

The difference of a user's heart rate from his/her average resting heartrate can be used as a personalized measure of how low his/her heart ratehas dropped. The PPG data can also be used to calculate the respirationrate of the user. Respiration rate often shows distinct changes withsleep and can be used as a feature to identify periods of sleep.Respiration can be calculated using a number of techniques: (i)measuring variabilities in the inter-peak intervals between heart beats,(ii) measuring slow variabilities (0.1-0.5 Hz) in the baseline of thePPG signal, and/or (iii) measuring slow periodic signals (0.1-0.5 Hz) inthe acceleration signal.

In one embodiment, data representing the heart rate, heart ratevariability, respiration rate, and/or oxygen saturation of the user ofthe wearable device 100 are calculated, and the resulting data is anadditional input for classifying the temporal periods or generating thebreathing disturbance score, according to embodiments of the presentdisclosure.

In various embodiments, the PPG sensors described herein may include oneor more electronic semiconductor light sources, such as LEDs, or otherlight sources that produce light using any of filaments, phosphors, orlaser. In some implementations, each light source of the wearable device100 emits light having the same center wavelength or within the samewavelength range. In other cases, at least one light source may emitlight having a center wavelength that is different from another lightsource. The center wavelengths of the light emitted by the one or morelight sources may be in the range of 495 nm to 570 nm. For example, aparticular green light source may emit light with a center wavelength of525 nm (or approximately 525 nm). In other embodiments, one or morelight sources may emit red light (e.g., 660 nm or approximately 660 nmcenter wavelength), and one or more light sources may emit IR light(e.g., 940 nm or approximately 940 nm center wavelength). In someembodiments, independent control of all light sources is provided. Inother embodiments, several light sources are controlled together as agang or bank.

The PPG sensors described herein may include one or more light detectorsadapted to detect wavelengths of light emitted from light sources,including those reflected or passed through elements that may impact thewavelengths or other aspects of the light. One or more PPG sensorsdescribed herein may include a single light source and a single lightdetector. Alternatively, other PPG sensors described herein may includemultiple light sources and/or multiple light detectors. A lightdetector, in an embodiment, may comprise one or more detectors fordetecting each different wavelength of light that is used by the lightsources. For example, a first detector may be configured to detect lightwith a wavelength of 660 nm (or approximately 660 nm), a second detectormay be configured to detect light with a wavelength of 940 nm (orapproximately 940 nm), and a third detector may be configured to detectlight with a wavelength of 525 nm (or approximately 525 nm). Examplesinclude photodiodes fabricated from semiconductor materials and havingoptical filters that admit only light of a particular wavelength orrange of wavelengths. The light detectors may comprise any of aphotodiode, phototransistor, charge-coupled device (CCD), thermopiledetector, or complementary metal-oxide-semiconductor (CMOS) sensor. Oneor more of the light detectors may comprise a bandpass filter circuit.

Breathing and/or sleep disturbances may be prevalent in the generalpopulation. For example, sleep apnea may affect 3-7% of adults. Despiteits prevalence, up to 80% of moderate and severe cases may beundiagnosed. Undiagnosed, sleep apnea is a risk factor in a variety ofother conditions including coronary artery disease and type II diabetes.Diagnosing sleep apnea requires the equipment of a sleep laboratory,which can be expensive, time-consuming and disturb natural sleep due tonumerous body-mounted sensors, which are part of the testing. Further,the severity of apnea can vary night to night and a fullcharacterization of sleep apnea severity may require multi-nightobservations, which are unsuitable for sleep laboratories. A continuous,comfortable sleep monitoring system would allow the general populationto screen for the risk of sleep apnea in their own homes withoutdisturbing natural sleep. Such a system would also allow users to trackthe severity of apnea over time and determine which, if any, lifestylechoices contribute to breathing disturbances during sleep, e.g.,obesity.

The disruption of normal breathing during sleep (or when the user isawake) can be detected in multiple ways using a variety of combinationsof sensors. In some embodiments, sleep apnea or other sleep/breathingconditions can be detected based on the presence of any combination ofthe following indicators: (i) reduction in blood oxygenation using pulseoximetry (e.g., as part of a PPG system utilizing PPG data); (ii)disruption of normal audible breathing patterns as measured by an audiosensor; (iii) change in respiration rate as measured using the PPGsystem; (iv) change in respiration using a strain gauge (e.g., wornaround the chest); (v) change in respiration using an accelerometer(that measures the periodic accelerations of respiration); (vi) changein respiration using a video camera that directly or indirectly observesrespiration; and/or (vii) change in respiration using a CO₂ sensor thatdetects changes in expelled carbon dioxide.

A combination of non-invasive sensors coupled with machine learningalgorithms can provide diagnostically relevant information. A devicewill be worn on a wrist wearable, which can sense blood flow usingmultiple optical wavelengths (red, infrared, and green) as well asacceleration due to motion. The green optical sensor enables monitoringof heart rate variability, and several measures of heart ratevariability have been linked to the presence of breathing disturbances.The red and infrared optical sensors provide information about the levelof blood oxygenation, which may decrease during apneas or hypopneas. Adifferent package of sensors on the bed to sense position, movement, orrespiratory effort may also be used in conjunction with the wristwearable system. An overview of this system is illustrated by system 200of FIG. 2 .

In some embodiments, all of the sensors used for detecting breathingdisturbances of a user are located on a wrist-worn device. For example,each sensor may be configured to generate sensor data from or near thewrist of the user. In one embodiment, the red PPG sensor 167 and the IRPPG sensor 168 are not located on the fingertip of the user. In anotherembodiment, the red PPG sensor 167 and the IR PPG sensor 168 are locatedon the fingertip of the user. In some embodiments, the sensor datagenerated by one or more of the sensors of the wearable device 100 isnot sufficient to calculate the Sp0₂ (e.g., to a medically acceptabledegree of accuracy) due to the location from which the sensor data isgathered (e.g., the wrist of the user). In other embodiments, the sensordata generated by one or more of the sensors of the wearable device 100(e.g., accelerometer 161) is sufficient to calculate the Sp0₂. (e.g., toa medically acceptable degree of accuracy). The motion sensor of thewearable device 100 is configured to generate motion data from alocation proximate to the location (e.g., adjacent, within the samehousing, from the same side of the wrist, etc.) from which the PPG datais gathered (e.g., by one or more of the PPG sensors 166-168).

In one example, a mattress or bed sensor is sitting on top of the user'smattress. In another example, the mattress or bed sensor is inside theuser's mattress. In yet another example, the mattress or bed sensor isunderneath the user's mattress. In some cases, the detection ofbreathing disturbances is performed without sensor data from a mattressor bed sensor. In some cases, none of the wearable device 100, clientdevice 170, and the server 175 is in communication with a mattress orbed sensor. Although a mattress or bed sensor is used as an example, anydirect respiration sensor could be used instead or in addition (e.g., achest strap inductance plethysmograph or an oronasal cannula).Additionally or alternatively, such a sensor may also generate heartrate and/or heart beat-to-beat interval, and the respiration data, heartrate data, and/or heart beat-to-beat interval data may be used in theclassification of a given temporal window and/or generation of abreathing disturbance score.

In some embodiments, sensor data (e.g., accelerometer data, PPG data,etc.) or other physiological metrics generated during an active periodduring which the motion data (e.g., accelerometer data) indicates thatthe user is active is used for detecting breathing disturbances of theuser during an inactive period during which the motion data indicatesthat the user is inactive (or in a sleep state). For example, if themotion data of a user generated during the user's active periodindicates that the user is physically active (e.g., exercisesfrequently, has a high step count, etc.), the baseline heart rate orother threshold that may cause a temporal window to be classified asbreathing disturbed may be increased (or decreased). Alternatively, ifthe motion data of a user generated during the user's active periodindicates that the user is not physically active (e.g., does notexercise frequently, has a low step count, etc.), the baseline heartrate or other threshold that may cause a temporal window to beclassified as breathing disturbed may be decreased (or increased).

In some embodiments, the classification of the temporal window and/orthe detection of the user's sleep state is based on the wristorientation data and/or the wrist movement data generated by the motionsensor of the wearable device 100 (e.g., accelerometer 161). Forexample, if the wrist orientation data indicates that the user's wristis oriented in a way that is inconsistent with sleep during a specificperiod, then the user may be determined not to be in a sleep stateduring the specific period. In another example, if the wrist movementdata indicates that the user is moving around, the baseline heart rateor other threshold that may cause a temporal window to be classified asbreathing disturbed may be increased.

In some embodiments, the PPG data generated by one or more of the PPGsensors of the wearable device 100 is filtered such that any PPG dataduring a specific period in which motion data (e.g., accelerometer data)indicates a high level of user movement is removed (or not consideredfor detecting breathing disturbances of the user). In some embodiments,only the PPG data generated during a period having motion data below athreshold is kept and utilized for detection of breathing disturbancesof the user.

Data from these different sensor streams are extracted using novelfeature extraction algorithms. Features related to heart ratevariability (e.g., calculated based on the sensor data from the greenPPG sensor 166), Sp0₂ or an indicator thereof (e.g., calculated based onthe sensor data from the red PPG sensor 167 and the IR PPG sensor 168),movement (e.g., accelerometer data), and/or respiration (e.g., sensordata from the mattress or bed sensor) are extracted each minute andprocessed by a classification algorithm to determine if the minute isbreathing disturbed or normal. Sensor data from the mattress or bedsensor and/or the accelerometer can be used to determine the onset andend of sleep. A high-level overview of the signal processing chain isillustrated by system 300 of FIG. 3 .

Features extracted from the sensor data may comprise various indicatorsused for classifying each temporal window as either breathing disturbedor normal. In some embodiments, these features may be extracted fromeach overlapping window of time, where each overlapping window of timeincludes a number of samples (e.g., sensor readings). In someembodiments, the extracted features include one or more of the mean ofthe samples, the standard deviation of the samples, the energy of thesamples, the maximum of the samples, the minimum of the samples, thedifference between the maximum and the minimum, the root mean square ofthe samples, the count of oscillations of the samples, the AC signal(e.g., high-frequency component) of the samples, the DC (e.g.,low-frequency component) signal of the samples, etc. In one embodiment,the extracted features are combined using a linear weighting function toform a discriminant value. For example, if the features for a temporalwindow are labeled as x1, x2, x3, etc., then the weighted sum iscalculated as w1x1+w2x2+w3x3+ . . . =d, where d is called thediscriminant value. This value is compared to a threshold whichseparates “breathing disturbed” epochs (or temporal windows) from“non-breathing-disturbed” epochs (or temporal windows). The threshold isdetermined using a set of training data where these labels have beenpreviously calculated by a human expert, or other means.

In some embodiments, a set of data is collected from a sleep lab where asleep expert has labeled the data, and the system creates, based on thecollected set of data, a dictionary of what the extracted features looklike in the normal case (e.g., during a non-breathing-disturbed temporalwindow) and the breathing disturbed case (e.g., during abreathing-disturbed temporal window). The system compares the featuresextracted from a given set of sensor data to the dictionary, anddetermines whether the extracted features are more similar to thosecorresponding to the normal case or those corresponding to the breathingdisturbed case.

The extracted features may be further processed and compared against aset of threshold values for determining whether a given temporal windowis breathing disturbed or not. For example, the DC signal of the sensordata from the red PPG sensor 167 divided by the DC signal of the sensordata from the IR PPG sensor 168 may be graphed, and any temporal windowhaving a relatively static value (e.g., between specific low and highthreshold values) for a threshold amount of time may be classified asbreathing disturbed. In some embodiments, the system may split thesignal generated by each of the red and IR sensors into amotion-sensitive component and a motion-insensitive component, and useonly the motion-insensitive component (and not the motion-sensitivecomponent) in classifying the temporal window as breathing disturbed ornot. In some embodiments, the system may split the signal generated byeach of the red and IR sensors into an arterial (or blood) oxygenationcomponent and a tissue (or other) oxygenation component, and use onlythe tissue oxygenation component (and not the arterial oxygenationcomponent) in classifying the temporal window as breathing disturbed ornot. In some embodiments, the system classifies the temporal window asbreathing disturbed based on detecting a pattern of low respiration(below a threshold level of respiration over a threshold amount oftime). In some embodiments, the system classifies the temporal window asbreathing disturbed based on detecting a spike in respiration (e.g.,indicative of a big gasp).

Other data collected from the motion sensors, PPG sensors, and othersensors may include: (i) measures of user movement; (ii) data forcalculating heart rate, heart rate variability, and respiration; and(iii) time-domain or frequency-domain measure of heart rate variability.The raw heart rate data can be used to define a number of higher-levelfeatures. For example: (i) rolling median or average heart rate onvarious timescales (e.g., over the last minute, last 5 minutes, etc.);(ii) rolling quantile of the heart rate distribution as measured onvarious timescales; and (iii) whether heart rate exceeded/went-belowsome threshold over the course of some interval.

Additionally, statistical features may be derived using time derivativeof heart rate. For example: (i) rolling maximum change in heart rateover various timescales; and (ii) rolling median, average, or some otherquartile of the distribution of heart rate derivative over varioustimescales. Similar statistical features can be derived using heart ratevariability or respiration. For example: (i) number of times HRV orrespiration events exceeded some threshold over various timescales; and(ii) rolling median, average, or some other quartile of the distributionof HRV or respiration over various timescales. Heart rate, thederivative of heart rate, heart rate variability, and respiration may benormalized or standardized appropriately on a per-user basis account fornatural variation across users. For example, a user's resting heart ratemay be subtracted from his or her instantaneous heart rate. Demographicinformation may also be used to renormalize features to account forvariation according to demographics.

These statistical features may be used to train a classifier to classifytemporal windows as either breathing disturbed or normal. In oneembodiment, the system uses a random forest classifier. Otherclassification methods include neural networks, hidden-Markov models,support vector machines, k-mean clustering, and decision trees.

In some embodiments, the machine-learning-based classifier system mayflexibly use or not use particular sensor inputs as they are available,without changing the underlying model. Thus, the same system could beused with a combination of wrist-worn sensors and external sensors, justwith the wrist-worn sensors, just with one or more PPG sensors and amotion sensor, or just with one or more PPG sensors. In someembodiments, the system uses all of the sensor data available. Forexample, if four sensors are generating (or generated) sensor data,sensor data from all four sensors are used to classify the temporalwindow as breathing disturbed or not. In another example, if three ofthe four sensors are generating (or generated) sensor data and one ofthe four sensors is not generating (or did not generate) sensor data,the sensor data from the three sensors are used to classify the temporalwindow as breathing disturbed or not.

In some embodiments, the system uses sensor data from at least one PPGsensor in classifying the temporal window as breathing disturbed or not.In some embodiments, the system does not use the sensor data generatedby the green PPG sensor and instead calculates HRV from the sensor datagenerated by the red PPG sensor, and uses the calculated HRV to classifythe temporal window as breathing disturbed or not. In some embodiments,the system uses the sensor data generated by the PPG sensor and not thesensor data generated by the red PPG sensor, the IR PPG sensor, or theaccelerometer. In some cases, the system uses sensor data generated by aGSR sensor to classify the temporal window as breathing disturbed or notand/or to generate a breathing disturbance score.

In some embodiments, upon determining that a first condition issatisfied, the system uses sensor data from a first combination ofsensors, and upon determining that the first condition is not satisfied,the system uses sensor data from a second combination of sensorsdifferent from the first combination of sensors. For example, in somecases, if the wearable device determines that the optical signal qualityis poor from the red and infrared PPG sensors, the wearable device doesnot extract features from the red and infrared PPG data and extractsfeatures (e.g., heart rate information or other indicators) only fromthe green sensor data. Alternatively or additionally, the wearabledevice may turn off the red and infrared PPG sensors upon determiningthat the optical signal quality is poor from the red and infrared PPGsensors (e.g., to conserve power and/or computing resources). Althoughthe red and infrared PPG sensors are used as examples, the techniquesdescribed herein can be extended to any of the sensors or anycombination thereof

In some embodiments, upon determining that a second condition issatisfied, the system uses a first classification rule for classifyingthe temporal window as breathing disturbed or not, and upon determiningthat the second condition is not satisfied, the system uses a secondclassification rule different from the first classification rule forclassifying the temporal window as breathing disturbed or not. In somecases, the wearable device may activate or deactivate one or moresensors based on the sensor data. For example, upon determining that theaccelerometer data indicates that the user has been moving around forthe past minute (or 5, 15, or 30 minutes, for example), the wearabledevice may turn off one or more sensors that are used for generatingsleep metrics (e.g., including breathing disturbance scores/metrics). Insome cases, upon determining that the accelerometer data indicates thatthe user has been moving around for the past minute (or 5, 15, or 30minutes, for example), the wearable device deactivates one or more redPPG sensors and one or more infrared PPG sensors. The wearable devicemay activate one or more deactivated sensors based on a determinationthat the user may be sleeping (e.g., based on a determination that theaccelerometer data indicates that the user has not been moving aroundfor the past minute (or 5, 15, or 30 minutes, for example).

Although one minute is used herein as an example temporal window forwhich this classification can occur, other temporal windows having adifferent duration may be used (e.g., 30 second temporal windows,5-minute temporal windows, etc.). In one embodiment, each temporalwindow has a fixed duration. In another embodiment, the duration of thetemporal window is adjusted over time based on the user's sleeping orbreathing patterns. For example, if the system (e.g., any combination ofthe wearable device 100, the client device 170, and the server 175)determines that the user's breathing disturbances frequently extendacross multiple temporal windows, the duration of the temporal windowsmay be increased.

In some embodiments, the system may automatically detect blocks of timeduring which the user of the wearable device 100 is (or was) asleep.Here, automatic detection may refer to a detection that occurs withoutan explicit instruction from the user, such as traversing on-screenmenus or otherwise inputting periods of time where the user is sleeping.It is to be appreciated that the term “automatic” does not preclude acase where the user may enable such a feature or act in a way that doesnot represent an explicit instruction of the timing when the usertransitions between different sleep states. Such an embodiment thatdetects blocks of time in which the user is (or was) asleep may obtain aset of features for one or more periods of time from motion data and/orPPG data obtained from a set of one or more motion sensors or dataderived therefrom. The system may then classify the one or more periodsof time as one of a plurality of statuses of the user based on the setof features determined for the one or more periods of time. The statusesmay be indicative of relative degree of movement of the user. In somecases, the statuses may be enumerated state values, such as active orinactive, where each state represent a different degree of movement orlikelihood that the user is in a sleep state. In some embodiments, onlythe motion data is used to detect the period during which the user is(or was) asleep. In other embodiments, sensor data from a combination ofthe motion sensor and one or more PPG sensors is used to detect theperiod during which the user is (or was) asleep.

The minute-by-minute performance of the classifier can be characterizedby a receiver operating characteristic (ROC) curve. An example ROC curve400 is shown in FIG. 4 . The epoch (or temporal window) classifieroutputs a probability of the minute being breathing disturbed. To make adecision about the minute, the system may apply a threshold such that ifthe output probability is above the threshold, the system declares theminute to be breathing disturbed, and otherwise, the system declares theminute to be normal (or not breathing disturbed). The value of thisthreshold is selected to tradeoff between false positives and misseddetections. For instance, a low threshold would have few misseddetections but may have many false positives. The determination of theprobability of the temporal window being sleep disturbed may be based ondemographic data. For example, the indicators and/or features describedherein may be weighted differently based on the demographic dataassociated with the user. The demographic data may include one or moreof the user's body mass index (BMI), age, neck circumference, weight,height, gender, and the like. In some cases, the probability may beincreased or decreased based on the demographic data associated with theuser (e.g., based on whether the user belongs to a specific category ofusers more (or less) likely to experience breathing disturbances). Forexample, if the user is 65 years old (or greater), has a BMI of 35 (orgreater), and has a neck circumference of 18 inches (or greater), theprobability of a given temporal window being sleep disturbed may beincreased (e.g., by adding an offset or multiplying by an offset).Alternatively or additionally, the threshold for determining whether agiven temporal window is sleep disturbed or not may be increased ordecreased based on the demographic data associated with the user (e.g.,based on whether the user belongs to a specific category of users more(or less) likely to experience breathing disturbances). In someembodiments, the specific user categories are generated and/or updatedbased on the demographics data and the classifications/breathing scoresgathered across a plurality of users.

The ROC curve of FIG. 4 captures this tradeoff by plotting the truepositive rate (sensitivity) against the false positive rate(specificity) over a range of thresholds. The example of FIG. 4illustrates the performance of a classification method based on PPG andaccelerometer features over 8500 minutes of sleep across several people.In some implementations, since most subjects will not have breathingdisturbances during sleep, the system selects or uses a threshold valuecorresponding to an operating point close to the vertical axis. Thiswould in effect sacrifice sensitivity (true positives) for specificity.In some cases, the system adjusts the threshold value based on sensordata or other metrics generated over time and/or based on user input orfeedback. In some cases, the threshold value is determined such that thefalse positive rate is fixed or is below a threshold level. In somecases, the threshold value is maintained and updated by a cloud service(e.g., server 175) and is downloaded (periodically or when the value isupdated) by the client device 170 and/or the wearable device 100 fromthe cloud service.

In some embodiments, the sensor data from different sensors are weighteddifferently. For example, each sensor may be assigned a fixed weightindicative of how much the sensor is to contribute to the classificationof the temporal windows as breathing disturbed or not. In anotherexample, the weights assigned to the sensors are adjusted based on thesensor data (e.g., the sensor data from the red PPG sensor and the IRPPG sensor may be weighted more heavily based on the sensor data fromthe green PPG sensor remaining constant or not changing more than athreshold amount for a threshold period of time).

The labels for each temporal window (e.g., breathing disturbed ornormal) are fed to a second classification algorithm to determine theseverity of breathing disturbance for a given sleep period (e.g., theentire night). This second classification stage may combine informationfrom the user's demographics (gender, weight, height, age) and theobserved night of sleep in its entirety to assign a breathingdisturbance score (e.g., severity of breathing disturbance) for theuser's night of sleep. The system may determine the percentage oftemporal windows that are classified as breathing disturbed and select ascore based on the percentage. In some embodiments, the system generatesa composite score for multiple nights, weeks, months, and/or years. Forexample, the threshold number of nights needed before the systemgenerates the composite score may differ based on the signal quality ofthe one or more sensors. For example, if the signal quality of one ormore sensors has been poor for the nights, the system generates thecomposite score after collecting more than 4 nights' worth of sensordata, classifications, and/or breathing disturbance scores. On the otherhand, if the signal quality of one or more sensors has been good, thesystem generates the composite score after collecting more than 2nights' worth of sensor data, classifications, and/or breathingdisturbance scores.

In one example, the breathing disturbance score is one of normal (e.g.,below 5%), mild (e.g., 5-10%), moderate (e.g., 10-20%), or severe (e.g.,20% or above). In another example, breathing disturbance score is one ofnormal (e.g., below 5%) or not normal (e.g., 5% or above). Suchinformation in some embodiments may also be used to provide a sleepfitness score, where higher breathing disturbance scores can correlatewith lower sleep fitness scores, along with data such as motion datadetermined using an accelerometer and the like. For example, a nightwith a normal breathing disturbance score and a within a normal range ofmotion for a user might receive a “normal” sleep fitness score, while anight with a not normal breathing disturbance score or higher thannormal amount of motion might receive a “not normal” sleep fitnessscore, using combinations of factors discussed herein as would beapparent to one or ordinary skill in the art in light of the teachingsand suggestions contained herein. Further, although example percentageranges are used in various examples, the percentage ranges may be set toother values and may be adjusted based on other historical data and/orphysiological metrics of the user and/or other users. In someembodiments, the system generates the breathing disturbance score onlyif the sleep period exceeds a threshold amount of time (e.g., 2, 3, 4hours). In some cases, the threshold amount of time may be adjustedbased on the signal quality of the one or more sensors. For example, ifthe signal quality of one or more sensors is poor, the system generatesthe breathing disturbance score only if the sleep period exceeds fourhours, and if the signal quality of one or more sensors is good, thesystem generates the breathing disturbance score as long as the sleepperiod exceeds two hours. Alternatively, the system generates thebreathing disturbance score regardless of how long the sleep period is.In some embodiments, the determination of the breathing disturbancescore may be based on demographic data. For example, the percentage ofthe temporal windows classified as breathing disturbed may be increasedor decreased based on the demographic data associated with the user(e.g., based on whether the user belongs to a specific category of usersmore (or less) likely to experience breathing disturbances). Thedemographic data may include one or more of the user's body mass index(BMI), age, neck circumference, weight, height, gender, and the like.For example, if the user is 65 years old (or greater), has a BMI of 35(or greater), and has a neck circumference of 18 inches (or greater),the percentage of the temporal windows classified as breathing disturbedmay be increased (e.g., by adding an offset or multiplying by anoffset). The specific user categories can be generated and/or updatedbased on the demographics data and the classifications/breathing scoresgathered across a plurality of users. For example, if the systemdetermines that a fewer number of users between age 65 and age 67 areexperiencing breathing disturbances, the example above may be updated sothat users of age 68 and older having a BMI of 35 or greater, and havinga neck circumference of 18 inches or greater would have their breathingdisturbance score adjusted upward (or have their probabilities ofindividual temporal windows being sleep disturbed adjusted upward).

In some embodiments, the system outputs one or more additional breathingdisturbance metrics. For example, the system may output one or more ofthe percentage of temporal windows classified as being breathingdisturbed, the number of breathing disturbances per hour (or per night),the apnea-hypopnea index (AHI), and a sleep apnea severity score (e.g.,normal for AHI below 5, mild for AHI of 5 or greater and below 15,moderate for AHI of 15 or greater and below 30, and severe for AHI of 30or above). The system may also output a graph illustrating the trend inthe breathing disturbance scores or metrics of the user over time (e.g.,daily, weekly, monthly, and/or yearly). In some embodiments, the systemmay output a recommendation that the user schedule a visit to a doctor.

In some implementations, upon determining that the breathing disturbancescore or metric is not normal (e.g., above a threshold percentage oftemporal windows classified as breathing disturbed or above a thresholdnumber of apnea events per hour), the system outputs a question alongwith a selectable option to gather additional information from the user.For example, the client device 170 may display the question “Are youdrowsy all the time?” with a user interface element for indicating “yes”or “no.” Upon detecting user selection of a predetermined option (e.g.,“yes”), the system outputs a recommendation for a sleep laboratory or adoctor for further examination. The system may select the sleeplaboratory or doctor from a list of sleep laboratories and/or doctorsregistered (e.g., as part of a revenue share program with the companyimplementing the system) and also in the user's geographical area. Insome cases, upon determining that the breathing disturbance score ormetric is not normal (e.g., above a threshold percentage of temporalwindows classified as breathing disturbed or above a threshold number ofapnea events per hour), the system outputs the breathing disturbancescore or metric for presentation to the user along with an advertisementfor a sleep laboratory, doctor, or medical device company (e.g., acontinuous positive airway pressure (CPAP) machine manufactured by amedical device company).

In some embodiments, the breathing disturbance scores generated for agiven user are stored in cloud storage and accessible to the user sothat the user can monitor the degree to which the user is experiencingbreathing disturbances over time and see how the user compares to otherusers of similar demographics (e.g., age, gender, height, weight,activity level, etc.). Both the user's own breathing disturbance scoresand the benchmarking data generated based on other users' breathingdisturbance scores will enable the user to work to decrease the severityof breathing disturbances though practices including CPAP therapy,weight loss, and/or changing sleeping orientation.

In some embodiments, upon determining that the user's breathingdisturbance score for a given night is one of mild, moderate, or severe,the system accesses the past scores stored in the cloud or on the user'smobile device and determines whether the user's breathing disturbancescore for the given night is unusual. For example, if the user's neverhad a score other than “normal” for the past 3 years, the system mayignore the score of “severe” generated for last night (e.g., once,twice, or for a threshold number of times) and output a score of“normal.” Alternatively, in such a case, the system may adjust the scoredown to “mild” or “moderate.”

In some embodiments, the breathing disturbance score or other breathingor sleep-related score, is presented in a graphical user interface (GUI)that also includes the benchmarking data generated based on other users'breathing disturbance scores. In some embodiments, the breathingdisturbance score is presented in a GUI that also includes the user'sweight, the user's activity level during the daytime, and/or the user'ssleeping orientation. In some embodiments, the breathing disturbancescore is presented in a GUI that also includes the history of the user'sbreathing disturbance scores/metrics. In some of such embodiments, thehistory spans at least 30 days. In some cases, the history spans atleast 365 days.

In some embodiments, the wearable device 100 includes a pressure sensorconfigured to sense relative motion of the wearable device with respectto the user's skin. For example, the pressure sensor generates sensordata indicative of how much force is exerted on the wearable device 100in a direction perpendicular to the user's skin (e.g., force pushing thewearable device 100 into the user's wrist) and/or in a directionparallel to the user's skin (e.g., force pushing the wearable device 100up and down the user's wrist and/or along the circumference of theuser's wrist).

In some embodiments, upon determining that a temporal window that wouldotherwise be classified as breathing disturbed includes pressure sensordata indicating that the force exerted in a direction perpendicular tothe user's skin is greater than a threshold level, the system classifiesthe temporal window as normal (or not breathing disturbed). In someembodiments, upon determining that a temporal window that wouldotherwise be classified as breathing disturbed includes pressure sensordata indicating that the force exerted in a direction parallel to theaxis of the user's forearm is greater than a threshold level, the systemclassifies the temporal window as normal (or not breathing disturbed).In some embodiments, upon determining that a temporal window that wouldotherwise be classified as breathing disturbed includes pressure sensordata indicating that the force exerted in a direction parallel to thetangential direction of the user's wrist is greater than a thresholdlevel, the system classifies the temporal window as normal (or notbreathing disturbed).

In some embodiments, upon determining that the pressure sensor datagenerated during a given time period indicates that the force exerted ina direction perpendicular to the user's skin is greater than a thresholdlevel, the system ignores other sensor data (e.g., PPG sensor dataand/or motion sensor data) generated during the given time period inclassifying the temporal windows as breathing disturbed or not. In someembodiments, upon determining that the pressure sensor data generatedduring a given time period indicates that the force exerted in adirection parallel to the axis of the user's forearm is greater than athreshold level, the system ignores other sensor data (e.g., PPG sensordata and/or motion sensor data) generated during the given time periodin classifying the temporal windows as breathing disturbed or not. Insome embodiments, upon determining that the pressure sensor datagenerated during a given time period indicates that the force exerted ina direction parallel to the tangential direction of the user's wrist isgreater than a threshold level, the system ignores other sensor data(e.g., PPG sensor data and/or motion sensor data) generated during thegiven time period in classifying the temporal windows as breathingdisturbed or not.

In one embodiment, the wearable device performs all the processing andsends the result (e.g., classifications and breathing disturbance score)to the mobile device or the centralized server. In another embodiment,the wearable device determines the classifications but the mobile deviceor server determines the breathing disturbance score. In yet anotherembodiment, a portion of the processing is performed on the wearabledevice, and the remaining portion of the processing can be performed onthe mobile device or server. In yet another embodiment, the sensor datais sent to the mobile device or server without any processing. In someof such embodiments, the sensor data is processed by the mobile deviceor server to determine the classifications and the breathing disturbancescore.

In some embodiments, the detection of breathing disturbances occurs on amobile device (e.g., the client device 170) that can communicate withthe wearable device. For example, the sensor data collected by thewearable device may be sent (with or without additional processing bythe wearable device) to such a mobile device of the user, and the mobiledevice can analyze the sensor data to classify the individual temporalwindows and generate a breathing disturbance score. The mobile devicemay wirelessly transmit the classifications and/or the breathingdisturbance scores or metrics to a cloud service (e.g., the server 175)for storage.

Advantageously, detection of breathing disturbances on such a mobiledevice can provide the power savings for the wearable device (e.g.,since detection is not performed by the wearable device), withoutrequiring the infrastructure of a centralized database.

In some embodiments, the detection of breathing disturbances occurs on acentralized server (e.g., the server 175) that can communicate with thewearable device and/or the mobile device associated with the user. Forexample, the sensor data collected by the wearable device may be sent(with or without additional processing by the wearable device) to such aserver (either directly or via the mobile device), and the server cananalyze the sensor data to classify the individual temporal windows andgenerate a breathing disturbance score. The centralized server maywireless transmit the classifications and the breathing disturbancescores or metrics to the mobile device and/or the wearable device forstorage and presentation to the user.

Advantageously, detection of breathing disturbances on such acentralized server can provide the power savings for the wearable device(e.g., since detection is not performed by the wearable device) and morerobust detection since more data and processing power may be availableon the server compared to the client device 170.

FIG. 5 is a flowchart illustrating an example method for monitoringbreathing disturbances of a user in accordance with aspects of thisdisclosure. The method 500 may be performed by the wearable device 100,the client device 170, and/or the server 175, either alone or in adistributed manner. The method 500 illustrates an example algorithm thatmay be programmed, using any suitable programming environment orlanguage, to create machine code capable of execution by a CPU ormicrocontroller of the wearable device 100, the client device 170,and/or the server 175. Various embodiments may be coded using assembly,C, OBJECTIVE-C, C++, JAVA, or other human-readable languages and thencompiled, assembled, or otherwise transformed into machine code that canbe loaded into read-only memory (ROM), erasable programmable read-onlymemory (EPROM), or other recordable memory of the wearable device 100,the client device 170, and/or the server 175 that is coupled to the CPUor microcontroller and then then executed by the CPU or microcontroller.

At block 505, the system receives sensor data generated by one or moresensors of the wearable device 100 worn by the user, wherein the sensordata is generated during a first period (e.g., a sleep period duringwhich the user is or was sleeping). The sensor data may include sensordata generated by any combination of the sensors described herein (orother data generated based on such sensor data). For example, the sensordata may include green PPG data generated by the green PPG sensor 166,red PPG data generated by the red PPG sensor 167, IR PPG data generatedby the IR PPG sensor 168, and/or motion data generated by theaccelerometer 161 (or other motion sensor described herein). Asdescribed herein, the sensor data may be generated by one or moresensors provided on the wearable device 100 and/or one or moreadditional sensors external to the wearable device 100.

At block 510, the system, for each respective temporal window of aplurality of temporal windows within the first period, classifies therespective temporal window as one of (i) breathing disturbed or (ii) notbreathing disturbed, based on the sensor data generated during therespective temporal window. For example, the temporal windows may eachbe 1-minute long, and the system, for each respective minute of aplurality of minutes within the sleep period, may analyze the sensordata generated by each of the sensors (e.g., one or more motion and PPGsensors) during that minute and determine whether the user is likely tohave been experiencing breathing disturbances during that minute. Inanother example, the temporal window is one of 30 seconds, 2 minutes, or5 minutes. The system may classify a given minute based on whether oneor more of the received sensor streams (e.g., each sensor streamgenerated by one of the sensors provided on the wearable activitytracker) include a feature indicative of the user having a breathingdisturbance. For example, upon determining that the green PPG sensordata generated by a green PPG sensor of the wearable activity trackerincludes a first period of slow oscillations followed a second period ofrapid oscillations during a given minute, the system may classify thegiven minute as breathing disturbed. The system may determine thelikelihood that the user is (or was) having a breathing disturbanceduring the given minute based on the similarity between the featuresextracted from the sensor streams and one or more predetermined featuresindicative of user having a breathing disturbance, and classify thegiven minute as breathing disturbed if the likelihood exceeds athreshold (e.g., 30%, 50%, 75%, etc. as illustrated by the ROC curve ofFIG. 4 ). If multiple sensor streams during the given minute includefeatures similar to the one or more predetermined features indicative ofuser having a breathing disturbance, the likelihood of the user having abreathing disturbance during the given minute may be higher. In someembodiments, prior to classifying the temporal windows, the system mayfirst determine a specific period during which the user is (or was)asleep, and classify each temporal window within the specific period,where the specific period is a subset of the first period for which thesensor data was received at block 505. In other embodiments, the systemclassifies each temporal window within the first period withoutdetermining whether the user is (or was) asleep during any portion ofthe first period. In some implementations, the system may utilizeinformation gathered outside the first period (or the specific periodduring which the user is determined to have been asleep). For example,the determination of whether a temporal window is breathing disturbed ornot may be based on the user's age, gender, height, weight, activitylevel, and/or other factors. In another example, the system determines abaseline heart rate (or other physiological metric such as Sp0 2, heartrate variability, respiration rate, etc.) associated with the user basedon the user's heart rate measured throughout the day (e.g., outside theperiod during which the user is determined to have been asleep) or theuser's heart rate over a number of days, weeks, months, or years, anduses the baseline heart rate (or other physiological metrics) todetermine whether a given temporal window should be classified asbreathing disturbed or not. For example, a spike in heart rate of 105may be indicative of a breathing disturbance for a user whose baselineheartrate is 50, but may not be indicative of a breathing disturbancefor another user whose baseline heartrate is 100.

At block 515, the system determines a breathing disturbance score basedon the classification of the plurality of temporal windows. For example,the system determines the percentage of temporal windows that wereclassified as breathing disturbed and generates the breathingdisturbance score based on the percentage. For example, the system mayselect the score from a set of available scores each associated with apercentage range (e.g., “normal” associated with a percentage range of0-5, “mild” associated with a percentage range of 5-10, “moderate”associated with a percentage range of 10-20, and “severe” associatedwith a percentage range of 20+). In some embodiments, the system maydetermine the breathing disturbance score based on the classificationand the information gathered outside the first period (or the specificperiod during which the user is determined to have been asleep). Forexample, the determination of the user's breathing disturbance score maybe based on the user's age, gender, height, weight, activity level,and/or other factors.

At block 520, the system outputs the breathing disturbance score forpresentation to the user. For example, the client device 170 may displaythe breathing disturbance score on the display screen of the clientdevice 170 along with a benchmark breathing disturbance score generatedbased on other users similar to the user. Such a benchmark breathingdisturbance score may be downloaded from the server 175. Additionally oralternatively, the client device 170 may receive a plurality of pastbreathing disturbance scores associated with the user from the cloudservice storing such past breathing disturbance scores, and display ahistory of breathing disturbance scores on the display screen of theclient device 170 based on the plurality of past breathing disturbancescores received from the cloud service. For example, the history mayinclude a graph illustrating a trend in the breathing disturbance scoresof the user. In some cases, additionally or alternatively, the systemmay transmit the breathing disturbance score to another device fordisplay on said another device.

In the method 500, one or more of the blocks shown in FIG. 5 may beremoved (e.g., not performed) and/or the order in which the method isperformed may be switched. For example, in some embodiments, one or moreof blocks 505, 510, 515 and 520 may be omitted. In some embodiments,additional blocks may be added to the method 500. The method 500 may beperformed by the wearable device 100, the client device 170, and/or theserver 175, either alone or in a distributed manner. For example, blocks505 and 510 may be performed by the client device 170 and blocks 515 and520 may be performed by the server 175. Alternatively, all of blocks505-520 may be performed by the server 175 such that the server 175receives the sensor data from the wearable device 100 or the clientdevice 170 at block 505 and outputs the breathing disturbance score tothe wearable device 100 or the client device 170 at block 520. Theembodiments of the present disclosure are not limited to or by theexample shown in FIG. 5 , and other variations may be implementedwithout departing from the spirit of this disclosure.

FIG. 6 is a flowchart illustrating an example method for monitoringbreathing disturbances of a user in accordance with aspects of thisdisclosure. The method 600 may be performed by the wearable device 100,the client device 170, and/or the server 175, either alone or in adistributed manner. The method 600 illustrates an example algorithm thatmay be programmed, using any suitable programming environment orlanguage, to create machine code capable of execution by a CPU ormicrocontroller of the wearable device 100, the client device 170,and/or the server 175. Various embodiments may be coded using assembly,C, OBJECTIVE-C, C++, JAVA, or other human-readable languages and thencompiled, assembled, or otherwise transformed into machine code that canbe loaded into read-only memory (ROM), erasable programmable read-onlymemory (EPROM), or other recordable memory of the wearable device 100,the client device 170, and/or the server 175 that is coupled to the CPUor microcontroller and then then executed by the CPU or microcontroller.For convenience, method 600 is described below as being performed by aserver.

At block 605, the server receives first PPG data generated by at leastone green PPG sensor, second PPG data generated by at least one red PPGsensor, third PPG data generated by at least one infrared PPG sensor,and accelerometer data generated by at least one accelerometer. Althoughaccelerometer data is used in the example of FIG. 6 , sensor datagenerated by one or more other motion sensors described herein may beused additionally or alternatively. In some embodiments, the serverreceives the data from the wearable device.

Alternatively, a mobile device receives the data from the wearabledevice (e.g., via Bluetooth or other wireless technologies), and themobile device wirelessly transmits the data to the server. Each of theat least one green PPG sensor, the at least one red PPG sensor, the atleast one infrared PPG sensor, and the at least one accelerometer may beprovided on a single wearable activity tracker configured to be worn ona wrist of a user. In some embodiments, the PPG data and/or theaccelerometer data may be raw data generated by the respective sensors.In other embodiments, the PPG data and/or the accelerometer data may beprocessed data generated by the wearable activity tracker based on suchraw data. For example, the processed data may include a set of featuresextracted from one or more of the first, second, and third PPG data andaccelerometer data (e.g., 17 features extracted from each minute's worthof sensor data). The set of features may be generated for individualtemporal windows (e.g., for every minute of the sensor data, for every 5minutes of the sensor data, etc.). In some embodiments, the sensors areprovided on the wearable activity tracker such that each of the sensorsare located on the same side of the user's wrist when the wearableactivity tracker is worn on the user's wrist. In some embodiments, atleast one sensor is not located on the same side of the user's wrist asthe remaining sensors. In some cases, one or more sensors external tothe wearable activity tracker are in communication with the mobiledevice and/or the server and wirelessly transmit additional sensor datato the mobile device and/or the server. For example, the mobile devicemay communicate with a bed sensor to collect ballistocardiogram dataindicative of the ballistic forces on the heart. In some cases,respiration data may be generated based on one or more of the first PPGdata, the second PPG data, the third PPG data, the accelerometer data,and/or the ballistocardiogram data.

At block 610, the server identifies, based on the accelerometer data (ora portion or subset of the set of features extracted from theaccelerometer data), a sleep period during which the user of thewearable activity tracker was asleep. In some embodiments, the serveridentifies the sleep period by determining which portion of theaccelerometer data indicates movement of the user that is consistentwith the user sleeping. For example, the server may identify the sleepperiod by determining which portion of the accelerometer data (or one ormore features associated with which timestamp or temporal window) isindicative of movement that is less than a threshold level, anddetermining the time period during which such portion of theaccelerometer data was generated as the sleep period. In some cases, theserver identifies the sleep period based on a determination that theportion of the accelerometer data that corresponds to the sleep perioddoes not indicate a threshold level of movement for at least a thresholdamount of time. Even when the user wearing the wearable activity trackeris sleeping, the wearable activity tracker may experience some movementand the accelerometer data may reflect such movement. In someembodiments, such movement while the user is sleeping is ignored by theserver, and the period during which such movement occurs is not excludedfrom the sleep period.

At block 615, the server classifies each minute of a plurality ofminutes within the sleep period as one of (i) a normal minute, or (ii) abreathing disturbed minute. The server may classify the individualminutes based on (i) the portion of the green PPG data generated duringthe sleep period, (ii) the portion of the red PPG data generated duringthe sleep period, (iii) the portion of the infrared PPG data generatedduring the sleep period, and any combination thereof (or a subset of theset of features extracted from the green PPG data, the red PPG data,and/or the infrared PPG data, for example, 5 features extracted from thegreen PPG data generated during the given minute, and 8 featuresextracted from the combination of the red PPG data and the infrared PPGdata generated during the given minute). The server may classify theindividual minutes further based on additional data corresponding to atime period outside the sleep period. For example, the determination ofwhether a minute is breathing disturbed or not may be based on theuser's age, gender, height, weight, activity level, and/or otherfactors. In another example, the server determines a baseline heart rate(or other physiological metric such as Sp0₂, heart rate variability,respiration rate, etc.) associated with the user based on the user'sheart rate measured throughout the day (e.g., outside the period duringwhich the user is determined to have been asleep) or the user's heartrate over a number of days, weeks, months, or years, and uses thebaseline heart rate (or other physiological metrics) to determinewhether a given minute should be classified as breathing disturbed ornot. For example, a spike in heart rate of 105 may be indicative of abreathing disturbance for a user whose baseline heartrate is 50, but maynot be indicative of a breathing disturbance for another user whosebaseline heartrate is 100. In some embodiments, the server generatesrespiration data based on one or more of the first PPG data, the secondPPG data, the third PPG data, the accelerometer data, and/or theballistocardiogram data, and classifies the individual minutes based onthe respiration data. For example, the server may classify a givenminute as breathing disturbed based on a determination that a portion ofthe respiration data corresponding to the given minute includes apattern of low values followed by a spike in value In some embodiments,the server classifies the individual minutes within a period duringwhich the sensor data indicates movement less than a threshold level anddoes not classify the individual minutes within another period duringwhich the sensor data indicates movement greater than the thresholdlevel. In some embodiments, the server classifies a given minute basedon a determination that the given minute is associated movement lessthan a threshold level. For example, the server may first determine thatthe given minute is associated movement less than a threshold levelprior to classifying the given minute.

At block 620, the server generates a breathing disturbance score for thesleep period based on the classification of the individual minutes. Forexample, the server determines the percentage of minutes that wereclassified as breathing disturbed and generates the breathingdisturbance score based on the percentage. For example, the server mayselect the score from a set of available scores each associated with apercentage range (e.g., “normal” associated with a percentage range of0-5, “mild” associated with a percentage range of 5-10, “moderate”associated with a percentage range of 10-20, and “severe” associatedwith a percentage range of 20+). In some embodiments, the server maydetermine the breathing disturbance score based on the classificationand the information gathered outside the sleep period (or the specificperiod during which the user is determined to have been asleep). Forexample, the determination of the user's breathing disturbance score maybe based on the user's age, gender, height, weight, activity level,and/or other factors. The server may update the benchmark breathingdisturbance score corresponding to the user's age, gender, height,weight, activity level, and/or other categories, based on the breathingdisturbance score of the user. The server may maintain past breathingdisturbance scores for a plurality of users each associated withrespective wearable activity trackers.

At block 625, the server causes the breathing disturbance score to berendered on a display screen of the mobile device. For example, theserver may wirelessly transmit the breathing disturbance score to themobile device, and the mobile device may display the breathingdisturbance score along with a benchmark breathing disturbance scorecorresponding to the user's age, gender, height, weight, activity level,and/or other categories. Such a benchmark breathing disturbance scoremay be downloaded from the server or another service. For example, themobile device may request from the server a benchmark breathingdisturbance score associated with a user category to which the userbelongs (e.g., any combination of user attributes such as “30-39 yearsold, male, 6′0″-6′3″, 175-185 lbs”), and the server may wirelesslytransmit the benchmark breathing disturbance score associated with theuser category. In some embodiments, how the individual minutes areclassified (e.g., as breathing disturbed or normal) is displayed in auser display. In one example, such user interface does not include anyclassifications of individual minutes outside the sleep period. Inanother example, such user interface may include classifications ofindividual minutes outside the sleep period.

In the method 600, one or more of the blocks shown in FIG. 6 may beremoved (e.g., not performed) and/or the order in which the method isperformed may be switched. For example, in some embodiments, one or moreof blocks 605, 610, 615, 620, and 625 may be omitted. In someembodiments, additional blocks may be added to the method 600. Themethod 600 may be performed by the wearable device 100, the clientdevice 170, and/or the server 175, either alone or in a distributedmanner. As one example, blocks 605 and 610 may be performed by thewearable device 100 or the client device 170, blocks 615 and 620 may beperformed by the server 175, and block 625 may be performed by theclient device 170. For example, by performing at least some of theprocessing on the wearable device 100, the amount of data that needs tobe wirelessly transmitted from the wearable device 100 to the clientdevice 170 and/or the server 175 can be reduced, thereby reducing thebandwidth requirements and/or reducing the latency associated with datatransfer/sync. Alternatively, all of blocks 605-625 may be performed bythe wearable device 100 alone, the client device 170 alone, or theserver 175 alone. For example, by performing at least some of theprocessing on the server 175, the amount of processing that needs to beperformed by the client device 170 can be reduced, thereby reducingpower consumption and/or computing resource consumption on the clientdevice 170. In some embodiments, no calculation for classifying thetemporal windows and/or generating the breathing disturbance score isperformed by the client device 170. In other embodiments, at least someof the calculation for classifying the temporal windows and/orgenerating the breathing disturbance score is performed by the clientdevice 170. The embodiments of the present disclosure are not limited toor by the example shown in FIG. 6 , and other variations may beimplemented without departing from the spirit of this disclosure.

FIG. 7 is a flowchart illustrating an example method for generating asleep fitness score for a user in accordance with aspects of thisdisclosure. The method 700 may be performed by the wearable device 100,the client device 170, and/or the server 175, among other such devices,alone or in a distributed manner. At block 702, user state data isreceived from a tracker device, although as mentioned in someembodiments the data can be received from one or more sensors, or othersuch sources, to a sub-system on the tracker device. In this example theuser state data includes PPG data, heart rate data, and motion ratedata, although various other types of data may be received as well, asmay include blood pressure data, eye movement data, audio data,ballistocardiogram data, body temperature data, and brain activity data,among other such options. As mentioned, this data can be obtained usingappropriate sensors or mechanisms on, or in communication with, thetracker in at least some embodiments. Any appropriate informationregarding the health, fitness, or state of the user may be obtainedwithin the scope of the various embodiments.

Once obtained, at block 704 the process determines at least one periodof sleep of the user. This can be determined as discussed elsewhereherein, as may use motion data in combination with any other relevantuser state data. The motion data can be obtained from an accelerometeror other motion sensor on the device, or from a camera or image sensor,among other such options. Also as mentioned, this can includedetermining multiple windows or periods of sleep over a night orspecified time period in some embodiments. For at least one period ofsleep, at block 706 the user state data is analyzed to generate sleepfitness data. As discussed elsewhere herein, this can involve using anyof the user state data to generate data useful for determining sleepfitness, as may involve determining a breathing disturbance score usingthe PPG data, a restlessness score using the motion data, a pulseregularity score using the heart rate information, etc. Any data usefulin determining a sleep state or fitness of the user may be calculated asdiscussed and suggested herein, and as would be apparent to one ofordinary skill in the art in light of the present disclosure. At block708 at least some of the generated or calculated sleep fitness data canbe compared against historical sleep fitness data. As mentioned, thiscan be used to determine whether any of the values deviate from thosetypically determined for a user, as different users will have differentamounts of motion or state variation during normal sleep patterns forthose users. At block 710 a sleep fitness score can be generated forthat user for the respective time period. As mentioned, this can bebased upon any or all of the sleep fitness data, as well as any variancefrom the historical values. For example, all normal values beingdetermined that correspond to historical averages for a given user,within an allowable variance range or threshold, might result in anormal fitness score being calculated for a user. This might result in ascore of “normal,” 100% for fit, etc. In some embodiments a weightedfunction of sleep fitness data can be used to generate a sleep fitnessscore, such as from 0 to 100, which can be presented to a user, with aseparate indication of whether this score is normal for that user. Ifthe score is outside a normal range, the score and indication may beaccompanied by additional information, such as may relate to the typesof data that resulted in the atypical score. The weightings of thevarious parameters of the function can be determined experimentally orthrough use of a trained machine learning algorithm, among other suchoptions. In some embodiments the sleep fitness score may be a simplescore of “fit” or “unfit,” among other such options, where anyindividual user state data outside the normal range may result in anunfit score. In various embodiments a sleep fitness score may be relatedto a single sleep fitness data point, such as may relate to a breathingdisturbance score or a restlessness score determined based on at leastdetected motion, while in various embodiments combinations of thefactors discussed and suggested herein are utilized to provide a moreaccurate and useful determination. Once generated, at block 712 theprocess can provide at least the sleep fitness score for presentation,and may also provide other relevant information as discussed andsuggested herein.

Traditional sleep analysis devices (those used by sleep labs or sensorsystems such as ARES™ Home Sleep Test), while more accurate, are onlyused by individuals who already know or suspect they have sleepdisorders. One advantage of using sensors on a wearable activity trackeris that sleep quality can be monitored for any individual for months andyears without disrupting his or her natural sleep routine. A combinationof wrist-worn sensors coupled with machine learning algorithms canprovide information relevant to sleep quality. Data from these differentsensor streams are extracted using novel feature extraction algorithms.

Sleep apnea diagnosis typically requires the equipment and personnel ofa sleep lab. A sleep technician monitors the subject while they sleepand records data from an array of sensors, such as, for example, brainactivity electroencephalogram (EEG) measurements, Sp0₂ via PPGmeasurements, airflow via nasal tube and pressure transducermeasurements, and/or electrical monitoring of heartbeats (ECG). In someimplementations, a finger Sp0₂ sensor coupled with an accelerometer andrespiration derived from a bed sensor can be used to monitor breathingdisturbances of a user. In other implementations, a wearable platformthat records Sp0₂ and heart rate through a forehead reflectance pulseoximeter and respiratory effort though a nasal pressure sensor may beused. These implementations may be expensive due to the equipmentrequired, the operation of which may not easily be available to thelayperson. Further, these implementations may require manual scoring ofthe data in order to come up with a medical diagnosis. Hence, a massscreening system implemented using an everyday wearable activity trackerwould enable people to determine whether they should seek furthermedical care with minimal cost compared to a sleep lab and withoutdisrupting their natural sleep routine.

In some embodiments, the wearable device 100 is designed to be wornevery day and is configured to provide, in addition to sleep metrics,other metrics or notifications (e.g., smartphone notifications, stepcount, jogging statistics, etc.). In such cases, the wearable device 100is therefore much more likely to be acquired by users who do not suspectthey have sleep apnea or other sleep/breathing conditions, and providesvaluable diagnostic information to the large number of users who do notyet know they have sleep apnea or other sleep/breathing conditions.

Information and signals disclosed herein may be represented using any ofa variety of different technologies and techniques. For example, data,instructions, commands, information, signals, bits, symbols, and chipsthat may be referenced throughout the above description may berepresented by voltages, currents, electromagnetic waves, magneticfields or particles, optical fields or particles, or any combinationthereof

The various illustrative logical blocks, and algorithm steps describedin connection with the embodiments disclosed herein may be implementedas electronic hardware, computer software, or combinations of both. Toclearly illustrate this interchangeability of hardware and software,various illustrative components, blocks, and steps have been describedabove generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure.

The techniques described herein may be implemented in hardware,software, firmware, or any combination thereof. Such techniques may beimplemented in any of a variety of devices, such as, for example,wearable devices, wireless communication device handsets, or integratedcircuit devices for wearable devices, wireless communication devicehandsets, and other devices. Any features described as devices orcomponents may be implemented together in an integrated logic device orseparately as discrete but interoperable logic devices. If implementedin software, the techniques may be realized at least in part by acomputer-readable data storage medium comprising program code includinginstructions that, when executed, performs one or more of the methodsdescribed above. The computer-readable data storage medium may form partof a computer program product, which may include packaging materials.The computer-readable medium may comprise memory or data storage media,such as random access memory (RAM) such as synchronous dynamic randomaccess memory (SDRAM), read-only memory, non-volatile random accessmemory (NVRAM), electrically erasable programmable read-only memory(EEPROM), flash memory, magnetic or optical data storage media, and thelike. The techniques additionally, or alternatively, may be realized atleast in part by a computer-readable communication medium that carriesor communicates program code in the form of instructions or datastructures and that can be accessed, read, and/or executed by acomputer, such as propagated signals or waves.

According to some embodiments, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices, wearable devices, or any other device thatincorporates hard-wired and/or program logic to implement thetechniques.

Processor(s) in communication with (e.g., operating in collaborationwith) the computer-readable medium (e.g., memory or other data storagedevice) may execute instructions of the program code, and may includeone or more processors, such as one or more digital signal processors(DSPs), general purpose microprocessors, ASICs, FPGAs, or otherequivalent integrated or discrete logic circuitry. Such a processor maybe configured to perform any of the techniques described in thisdisclosure. A general purpose processor may be a microprocessor; but inthe alternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, for example, acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Accordingly, the term“processor,” as used herein may refer to any of the foregoing structure,any combination of the foregoing structure, or any other structure orapparatus suitable for implementation of the techniques describedherein. Also, the techniques could be fully implemented in one or morecircuits or logic elements.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wearable device, a wirelesshandset, an integrated circuit (IC) or a set of ICs (e.g., a chip set).Various components, or units are described in this disclosure toemphasize functional aspects of devices configured to perform thedisclosed techniques, but do not necessarily require realization bydifferent hardware units. Rather, as described above, various units maybe combined in a hardware unit or provided by a collection ofinter-operative hardware units, including one or more processors asdescribed above, in conjunction with suitable software and/or firmware.

Although the foregoing has been described in connection with variousdifferent embodiments, features, or elements from one embodiment may becombined with other embodiments without departing from the teachings ofthis disclosure. However, the combinations of features between therespective embodiments are not necessarily limited thereto.

1.-20. (canceled)
 21. A computer-implemented method, comprising:determining a sleep period of a user based on motion data of the usercaptured by one or more motion sensors; determining, for the sleepperiod of the user, breathing disturbance data based onphotoplethysmographic (PPG) data captured by one or more PPG sensors;calculating a sleep fitness score associated with the sleep period,based on a classification of a plurality of temporal windows within thesleep period as representing disturbed breathing for the user or notbreathing disturbed for the user, wherein the classification of atemporal window among the plurality of temporal windows is determinedbased on a baseline heart rate of the user and the breathing disturbancedata; and providing, for presentation on a display device, at least oneof the sleep fitness score or the breathing disturbance data.